Image tagging is useful for organizing, managing, and searching images. Custom tagging is a specialized type of image tagging, in which images are tagged with custom classes determined by a user. Custom classes may be fine-grained classes of a general object category. For example, a user may define custom classes such as basket-ball shoes, gym shoes or football shoes for a general object category of shoes. Custom classes may also be customized for specific use cases, such as a business user who tags assets for day-to-day operations and management. For example, the business user may tag images for a general object category of automobiles with custom classes such as model numbers and various points of view, such as top-view, side-view, rear-view, interior, and exterior. Custom tagging may also improve stock or inventory search capabilities based on custom queries associated with custom classes.
Although images may be tagged manually by a person or a group of people, it is more convenient to utilize a device trained with machine learning to tag images automatically, particularly a large volume of images. A device trained with machine learning is useful so long as the device is able to perform this function accurately and within a reasonable time.
Even with automation, custom tagging is very challenging for fine-grained classification when the difference between objects is subtle, such as a minor cosmetic difference in appearance. For example, automobiles having the same make but different model numbers may have similar exterior appearances, thus making it difficult to distinguish custom classes of the different models. Another challenge for custom tagging is apparent when multiple objects are present within a single image. Conventional approaches for custom tagging are performed by focusing on images containing one type of object, such as cars, birds, or dogs. These conventional approaches for custom tagging fail to identify the correct classes when the models are applied to images containing different kinds of objects.